Internal cues for optimizing reproduction in a varying environment

In varying environments, it is beneficial for organisms to utilize available cues to infer the conditions they may encounter and express potentially favourable traits. However, external cues can be unreliable or too costly to use. We consider an alternative strategy where organisms exploit internal sources of information. Even without sensing environmental cues, their internal states may become correlated with the environment as a result of selection, which then form a memory that helps predict future conditions. To demonstrate the adaptive value of such internal cues in varying environments, we revisit the classic example of seed dormancy in annual plants. Previous studies have considered the germination fraction of seeds and its dependence on environmental cues. In contrast, we consider a model of germination fraction that depends on the seed age, which is an internal state that can serve as a memory. We show that, if the environmental variation has temporal structure, then age-dependent germination fractions will allow the population to have an increased long-term growth rate. The more the organisms can remember through their internal states, the higher the growth rate a population can potentially achieve. Our results suggest experimental ways to infer internal memory and its benefit for adaptation in varying environments.


Introduction
Organisms can adapt to a varying environment by having diversified traits among their offspring.A common form of such diversity is dormancy, where some individuals enter a dormant state, while others remain active [1][2][3].Those that are active will contribute to the growth of the population under good environmental conditions, but will be vulnerable to periods of harsh conditions.On the other hand, the dormant individuals are often tolerant to environmental stress and thus help preserve the lineages during harsh periods.For example, in a bacterial population, while most cells grow and divide normally, some cells randomly switch to a reversible dormant state called persister cells, which makes them tolerant to antibiotics when normal cells would perish [4][5][6].Other examples include seed dormancy in plants, dauer larva in nematodes, diapause in insects, etc. [1,3,7,8].These are thought to be a strategy known as diversified bet-hedging [9,10], in which organisms express different traits with some probability to create diversity in the population, so as to increase the long-term growth rate of the population under environmental variations [11][12][13].
In the simplest form, bet-hedging organisms have fixed probabilities of expressing different traits [11].But more generally, organisms can sense cues from the environment that will influence these probabilities [14,15].Such cues may be indicative of future environmental conditions, so that the organisms may bias the probabilities towards traits that are favourable in the likely environment.It has been shown that the information contained in the cue about the environment will contribute to an increase in the population growth rate [14,16,17].However, sensing and responding to environmental cues may come at a cost, as it requires the expression of specific sensors and signalling mechanisms [18].Besides, there may not be enough time for the organisms to respond to the cues through phenotypic plasticity, as the environment may have changed by the time the trait is developed [19,20].Therefore, it is not always beneficial to rely on environmental cues.
Besides external signals, the behaviour of organisms can be influenced by their internal states, such as physiological or metabolic states [21].One example is the reserve level-a starved animal may choose to forage more aggressively despite higher predation risk [21,22].Another example is the age of the organism-it is known that the age of seeds can affect germination in annual plants [23].These internal states are not sensors that directly measure the external environment.However, they may become correlated with the environment as a result of selection, because certain states are associated with higher fitness in past environmental conditions and thus become more common in the population.Therefore, the distribution of such internal states among the population can potentially provide information about the past environment, which may be utilized by the organisms to predict future conditions if there is temporal correlation in the environment.We will study an example of this situation and show that internal states of the organisms can indeed serve as internal cues to help them adapt to varying environmental conditions.Such internal states effectively provide a memory about the past outcomes of selection, which may help predict the future environment.Moreover, we show that a larger memory capacity enables higher gains in the population growth rate.Our results suggest that internal states that were not developed for sensing the environment could nevertheless be co-opted as internal cues for adaptation, which would save the cost of sensors and may thus be a more efficient strategy.
To study adaptation in varying environments, we will use seed dormancy as our main example.Seeds of annual plants will either germinate or stay dormant in a given year.While dormancy sacrifices the short-term fitness of the seeds, it preserves the population from a catastrophically bad year with very low yield and thus results in higher long-term benefit.This has been studied as a classic model of bet-hedging [11,14], supported by the fact that dormant seeds eventually germinate under similar environmental conditions [23] and that the germination fraction is negatively correlated with local environmental variability [24].It is known that germination is influenced by environmental cues such as temperature, humidity and the number density of surrounding seeds [15,25].Moreover, there is evidence that the probability a seed will germinate also changes with the age [23,26,27].However, the adaptive value of such age dependence in germination has not been fully studied [3,28].It was shown in [28] that the evolutionarily stable probability of germination does not depend on seed age if there is no density dependence.Yet, their model did not include temporal correlation in the environmental variation, which is crucial for memory to be useful in predicting future environments [29][30][31][32][33].We will show that, when there is temporal structure in the environmental variation, age-dependent germination probabilities can increase the long-term growth rate of the seed population.

Background (a) Cohen's model of seed dormancy
Let us first briefly review the idea of bet-hedging and how information emerges as a central quantity in determining the long-term growth rate of the population.We will follow the classic model of seed dormancy in annual plants by Cohen [11,14], as illustrated in figure 1a.Each year can be 'good' (denoted as environment 1 ¼ 1) or 'bad' (1 ¼ 0) for the plant.Seeds that germinate ('phenotype' ϕ = 1) in a good year will be able to grow and produce a large number (Y 1 ) of new seeds.However, in a bad year, germinated plants will have a low yield (Y 0 ).We will set Y 0 = 0 and denote Y 1 = Y for simplicity, meaning that germinating in a bad year will result in no offspring.All germinated plants perish at the end of the year, regardless of their yield.Seeds that stay dormant (ϕ = 0) will remain viable the next year with probability V. Thus, the fitness of a seed in a given environment can be summarized by the matrix In addition, we assume that the number of consecutive good years follows a geometric distribution, whereas that of bad years has a narrow distribution (figure 1b and appendix §A(b)).This is meant to describe the scenario where good growth conditions are disrupted by random occurrence of disasters that affect the growth for a characteristic number of years.
In the simplest case where seeds receive no environmental cues, the fraction of seeds that germinate each year is assumed to be a constant, denoted by q.In a good year, the total number of seeds will grow by a factor (1 − q)V + qY, whereas in a bad year, the number of seeds will reduce to only a fraction (1 − q)V of the previous year.The long-term growth rate of the population will be given by (see derivation in appendix §A(a)) where p is the frequency of good years and (1 − p) is that for bad years.The germination fraction that maximizes the long-term growth rate is for p > V/Y and 0 otherwise.In the limit of high yield (Y ≫ V), this leads to the classic result q* ≈ p, which means the optimal germination fraction should match the frequency of good years [11].The model can be extended to seeds that receive some external cue (ξ) about the environment [14].In this case, the optimal germination fraction will depend on the cue.As a result, the population can grow faster than without the cue (appendix §A(a)).These well-known results are summarized schematically in figure 2a.At the top level is the maximum possible growth rate L max , which is attainable only if individuals have perfect information about future environmental conditions and respond accordingly, i.e. germinate if it will be a good year and go dormant if it will be bad.On the other hand, if there is no environmental cue, the best strategy is bet-hedging with fixed probabilities, which achieves a growth rate L bet .This is less than L max by an amount Hð1Þ, which is the Shannon entropy from information theory that quantifies the uncertainty of the varying environment (appendix §A(a)).However, if a cue ξ is used to help predict the environment, the population can increase the growth rate from L bet to L cue , up by an amount Ið1; jÞ that is equal to the mutual information between the cue and the environment (appendix §A(a)).Note that L cue is still not as high as L max unless the cue is fully accurate.The relations between these growth rates illustrated here (similar to plots in [16,34]) royalsocietypublishing.org/journal/rspb Proc.R. Soc.B 290: 20230096 show that, in order for the population to better adapt to varying environments, it must utilize available sources of information about the environment.

(b) Internal source of information
Instead of sensing external cues, below we consider another possibility for organisms to use their internal states as a source of information.We will use the age of seeds as an example.The state diagram representing seed ages are illustrated in figure 1c, where a state s a represents a seed of age α.A blue arrow represents a seed going into dormancy for 1 year, so that the age is increased by 1.An orange arrow represents a seed that germinates and potentially produces new seeds, which will have age 0. The weights on the arrows represent the probability of germination or dormancy.For a simple bet-hedging strategy without any cues, the probability of germination will be a constant, which equals q* from equation (2.2), independent of the seed age.We will study the case where the germination fraction can depend on the seed age and show that the population can acquire information from this internal state to achieve a higher growth rate.

(a) Seed age as an internal cue
We first study whether the seed age as an internal state contains useful information about the environment.Let α t−1 be the seed age at the beginning of year t and 1 t be the coming environment that year.If α t−1 has no information about the environment, then it will be statistically independent of 1 t , i.e.Pð1 t ja tÀ1 Þ ¼ Pð1 t Þ.Therefore, whether seed age is informative about the environment can be inferred from the conditional probability Pð1 t ja tÀ1 Þ.To calculate that, we simulate a sufficiently long sequence of environments, denoted by 1 t for each year t.We also simulate a single lineage of plants that uses the constant germination fraction q*.Each year, the seed can either germinate or stay Figure 2. The long-term growth rate L of populations with different sources of information.(a) The value of external cues: organisms can respond to environmental cues through phenotypic plasticity.L max is the maximum possible growth rate attainable if the population has perfect information about the future environment.L bet is the highest growth rate achievable by a bet-hedging population without receiving cues, which is suppressed by the entropy of the environment Hð1Þ.L cue is the growth rate when the population utilizes a cue ξ that has a mutual information Ið1; jÞ with the environment.(b) The value of internal memory: organisms can utilize their internal states as indirect cues to adjust their behaviour.L bet from bet-hedging also represents the case with no memory, which corresponds to having only one internal state (L = 1).More internal states (L > 1, see figures 5 and 6) can provide more information about the future environment, Ið1 t ; a tÀ1 Þ, and allows a higher growth rate L int for the population.L mem is the highest growth rate achievable by organisms with a perfect memory (L → ∞) of their lineage history.Memory is beneficial provided that the past environment contains information about the future environment, i.e.Ið1 t ; f1 s g s,t Þ .0.
royalsocietypublishing.org/journal/rspb Proc.R. Soc.B 290: 20230096 dormant, and the probability of choosing the phenotype ϕ t is further weighted by the fitness f ð1 t , f t Þ to account for selection (see procedure in appendix §A(c)).The seed age along the lineage is recorded as α t .From the sequences of 1 t and α t , we estimate the joint probability distribution Pð1 t , a tÀ1 Þ, from which the conditional probability Pð1 t ja tÀ1 Þ is calculated.As shown in figure 3, the probability of the environment 1 t does depend on the seed age α t−1 .This means that knowing the seed age allows a more accurate prediction of the coming environment.Therefore, it is possible for the population to 'co-opt' the seed age as an 'internal cue' for the environment.In analogy to the case of external cues, we expect that such information can be used to increase the long-term population growth rate.We therefore consider a strategy where the germination fraction depends on the seed age, denoted by q a and represented by weights on the arrows in figure 1c.To calculate the long-term growth rate, let N be a vector that represents the age-structured population, with components N a being the number of seeds of age α.The dynamics of N is described by a matrix Mð1; qÞ that depends on the environment 1 and the germination fractions q (with components q a ), Each year, the population vector is multiplied by the matrix that corresponds to the current environment 1 t , Here, Mð1 t ; qÞ is a random matrix because 1 t is a random variable.The temporal sequence of 1 t is randomly drawn according to the distributions of good and bad years.The long-term growth rate L of the population is then given by the Lyapunov exponent of the product of these random matrices [35], which is calculated numerically (see methods in appendix §A(b)).
We vary the age-dependent germination fractions q a to maximize L. As expected, this growth rate using seed age as an internal cue (L int ) is greater than that of bet-hedging without cues (L bet ), as illustrated in figure 2b (figure 6).The optimal germination fraction as a function of seed age is shown in figure 4.An intuitive explanation for the age dependence is that, in this example, the bad environment typically lasts a number of years, so it is advantageous for a seed to stay dormant for a similar period of time to wait it out.Those that germinate in the wrong phase of the bad year cycle will be eliminated by selection, and the remaining individuals tend to be synchronized with the environment.In contrast, if there is no temporal structure in the environment, such as when the environment is randomly and independently chosen each year, then the seed age will no longer be correlated with the environment.In that case, the best strategy is to have a constant germination fraction (equal to q* in the bet-hedging case, see figure 4), as argued in [28].
Note that the information about the environment is contained in the distribution of seed ages within the population, which results from selection in previous years.Compared to the case of an external cue that is shared by all individuals, the seed age varies among individuals (which prevents an analytical expression for L).It acts as an individual's memory of its own lineage history, which helps it infer the likely environment in the future.Importantly, the increase in the population growth rate does not come at any cost associated with sensing external cues.Thus, such an internal source of information proves to be beneficial for the population.

(b) Internal states as memory
We have shown that internal states of organisms may help them 'remember' the past outcomes of selection to be able to predict the future environment, leading to an increased population growth rate.Intuitively, the more the organisms bet-hedging age dependent 0.8  1c has potentially an infinite number of states.They can be truncated at a finite number L, such that seeds exceeding age (L − 1) will remain in the state s L−1 until they germinate or perish (figure 5a).This allows us to study how the population growth rate depends on the number of states L.
We first note that having only one internal state (L = 1, figure 5b) is effectively having no memory, because the system will always be in that same state regardless of the past events.In this case, the germination fraction is always equal to q 0 associated with the only state s 0 .Having a constant germination fraction means that this case corresponds to the simple bet-hedging strategy.The maximum longterm growth rate will just be L bet achieved at q 0 = q* found in equation (2.2).
For two internal states (L = 2, figure 5c), the model reduces to 'phenotypic switching', in which the organisms randomly switch between two phenotypes (germination or dormancy) with fixed transition probabilities.Specifically, the probability for a dormant seed to germinate next year is q 1 , and the probability for a new seed (that came from a germinated plant) to go dormant is 1 − q 0 .This is a Markov process, for which the transition between phenotypes does not depend on how long a phenotype has lasted.It implies that the germination fraction only depends on whether the seed is fresh (age 0) or has been dormant (age > 0), but not on how long it has been dormant.As a result of being Markovian, the duration of the dormant phenotype will be geometrically distributed.
A larger L will allow the germination fraction to depend more sensitively on the seed age (L > 2, figure 5a).The number of states L roughly represents how many dormant years a seed can remember.For each number L, we search for the maximum long-term growth rate L over the parameters {q 0 , …, q L−1 } (see methods in appendix §A(b)).As shown in figure 6, L increases monotonically as more states are incorporated.Therefore, more memory allows faster population growth and hence better adaptation to environmental variation.Note that L quickly approaches a limit L mem when L becomes greater than the typical duration of the bad environment (equal to 5 in this example, see figure 1b).Intuitively, there is no need to remember longer dormancy because there is no benefit in staying dormant for longer than the duration of bad years.The relation between the growth rate and memory is illustrated schematically in figure 2b.
If we think of seed age as an internal cue for the environment, we can calculate the mutual information Ið1 t ; a tÀ1 Þ between the environment 1 t and the seed age α t−1 , using the joint probability Pð1 t , a tÀ1 Þ calculated the same way as in §2(b).Figure 6 shows that the mutual information also increases with the number of states L, as more memory is available.When plotted against each other, the long-term growth rate L increases with the mutual information I (figure 6 inset), just like for an external cue.Note that in Cohen's model with external cues [14], L is simply proportional to I (see equation (A 14) in appendix §A(a)).In comparison, for the same amount of information I, the population achieves a higher growth rate L using seed age as an internal cue (figure 6 inset).

(c) Other forms of memory
So far we have considered a very specific structure for the state diagram (figure 5a, 'age diagram').In general, the state diagram is a mathematical representation of memory in a history-dependent dynamical process (known as the 'e-machine' of a stationary stochastic process [32,36]).It might be possible that, given the number of internal states, there are other diagrams that can lead to a higher longterm growth rate.Such diagrams could represent other types of internal states instead of the age.For example, the reserve level of an organism can be represented by a linear diagram, such that the organism moves up one or more states if it succeeds in foraging or moves down one state if royalsocietypublishing.org/journal/rspb Proc.R. Soc.B 290: 20230096 it fails [21].To find which structure of internal states provides the highest long-term growth rate for the population, we search all possible diagrams of a given number of states (up to L = 6, beyond which it is computationally difficult), optimizing the weights q a for each diagram (appendix §A(d)).
It turns out that the age diagram in figure 5a is optimal for the temporal structure of the environment that we assume (figure 1b), which means age dependence is the optimal form of memory in this case.The benefit of memory for adaptation in varying environments has been previously studied in different models that sometimes involve transgenerational phenotypic memory (e.g.[29,31]).Many such models were restricted to simple forms of memory, such as a Markov process for discrete variables (as in [29]) or an autoregressive process for Gaussian variables (as in [31]), which both have exponentially decaying autocorrelation functions (hence geometrically distributed durations of each phenotype, see below).Our description in terms of internal states and state diagrams provides a more general representation of memory that can capture any temporal structure (as in [32]).By searching different state diagrams earlier, we have effectively compared different forms of memory (under a constraint on complexity as measured by the number of internal states).Note that most of the diagrams would represent a memory that is partially transgenerational.The age diagram that we have focused on is special in that the memory is within the lifetime of an individual, because all arrows that represent germination and production of a new seed (orange arrows in figure 5) point to the same state in the diagram, thus making it memoryless of what happened before.Therefore, the form of memory considered in our model is motivated by the observed age dependence of germination and does not require mechanisms of transgenerational epigenetic inheritance.
We expect that memory is potentially useful provided that the environmental variation is temporally structured, i.e. there is positive mutual information between the past and future environments, Ið1 t ; f1 s g s,t Þ [32].In particular, in the special limit of extreme selection where the fitness matrix is diagonal (appendix §A(e)), the benefit of memory as measured by the gain in long-term growth rate (the difference between L mem and L bet in figure 2b) is exactly given by this mutual information.For a more general case, such as the fitness matrix in Cohen's model used in most of our results, positive mutual information would be necessary but not sufficient for memory to be beneficial overall.Furthermore, the benefit of memory may also depend on how the trait is influenced by the environment (which may or may not be adaptive).In our model, for example, the germination fraction q may depend on both the seed age α and an environmental signal ξ (as in §2(a)).A detailed treatment of such possibilities is beyond the scope of this article.

Discussion (a) Characterization of internal memory
Memory arising from age-dependent germination fractions can be characterized by the distribution of the duration of dormancy.That is, given a large number of fresh seeds, what is the distribution of the time that each seed stays dormant before germinating.To calculate this distribution, we simulate one lineage of seeds over a long time in the absence of selection (appendix §A(c)), and record the sequence of phenotypes, i.e. whether a seed germinated or not each year.Figure 7 shows the distribution of the number of consecutive years that successive seeds germinate or that a seed stays dormant.The number of consecutive germination years is geometrically distributed with a mean of 1/q 0 (figure 7a), because every new seed has the same probability q 0 of germinating.In other words, a new seed has no memory of the age of the plant that it came from.Thus, the absence of phenotypic memory is signified by the geometric distribution.
On the other hand, the distribution of the consecutive dormant years (i.e. the duration of dormancy) depends on the number of internal states L. For L = 2, as discussed in §4(b), there is no memory of how long a seed has been dormant.Indeed, the distribution of dormancy durations is geometric with a mean of 1/(1 − q 1 ) (figure 7b).But as L increases, the distribution becomes more bell shaped and closer to the distribution of consecutive bad years (figure 1b).(In the limit where the fitness matrix f 1f is diagonal, the optimal strategy will be such that the duration of each phenotype exactly matches the distribution of the corresponding environment; appendix §A(e).)The deviation of the distribution from being geometric indicates that the seed has memory of how long it has been dormant, which is necessary for the germination fraction to depend on the .The distribution of the duration of consecutive germinations or dormant years along a lineage of seeds.Different colours correspond to age-dependent germination fractions q a for different memory capacities L. (a) For each L, the duration of germinations matches a geometric distribution with a mean of 1/q 0 (dashed line for L = 2 and solid line for L = 10), meaning that there is no memory of previous germinations.(b) The duration of dormancy has a distribution that changes shape depending on the memory capacity L. L = 2 ( phenotypic switching) results in a geometric distribution with a mean of 1/(1 − q 1 ) (dashed line).Larger L results in deviation from a geometric distribution, which is indicative of having internal memory.
royalsocietypublishing.org/journal/rspb Proc.R. Soc.B 290: 20230096 seed age.Thus, the shape of the dormancy distribution can be used as an experimental signature of internal memory.
The best demonstration of memory in phenotypic changes is found in experiments on the bacteria Bacillus subtilis [37].During its growth, B. subtilis can switch between two phenotypes, either as a free-moving cell by making flagella or as part of an aggregate by producing extracellular matrix [38,39].It is thought that the aggregate cells have an advantage for colonization and can better cope with a harsh environment by sharing resources, whereas the motile cells are better at dispersing and searching for nutrients.The durations of these two cell types along continuous cell lineages are measured in a constant environmental condition [37].It was found that the time a lineage stays in the motile cell type follows an exponential distribution with a mean of ∼81 generations, while the aggregate cell type is maintained for a narrowly distributed duration with a mean and standard deviation of 7.6 ± 2.1 generations (see fig. 2d,f of [37]).This implies that the motile cell type is memoryless, while the aggregate cell type has memory.That is, an aggregate cell keeps track of how long it has been part of an aggregate, whereas a motile cell turns off motility with a fixed probability at every cell division.These two distributions of phenotype durations look similar to those found in our model (figure 7).Importantly, since the switching of cell types is measured in a constant environment, it is evident that the phenotypic changes are influenced by some internal states of the cell, rather than external cues.This method of inferring the existence of internal memory by measuring the duration of phenotypes can be potentially applied to seeds.It would require measuring the duration of seed dormancy by planting fresh seeds in separate pots under the same environmental condition and recording how soon they germinate.The distribution of dormancy duration measured this way could depend on the environmental condition (such environment-dependent memory has been observed in microbial experiments [40]).Therefore, the existence of memory should be tested in each environmental condition.

(b) Evidence for age-dependent dormancy
Our model assumes that the probability of a seed entering or exiting dormancy depends on the age.If the bad environment typically persists for a number of years, then the model predicts that the probability of exiting dormancy should be small initially and increase over a timescale that matches the duration of bad years (figure 4).Data from past experiments have shown that for different species the germination fraction can either increase or decrease between the first and second years [23], while data going beyond the second year are scarce.To test the aforementioned prediction also requires knowing the statistics of bad years.Alternatively, age-dependent germination can be tested by measuring the distribution of dormancy durations, as discussed in §4(a) (figure 7b).For that purpose, one has to measure the final age of seeds right before they germinate.Studies on seed age structure have been done in the past [26,27], but with the goal of measuring the current age of seeds in a population at a given time, even though some seeds will continue to be dormant.We are not aware of existing studies that have measured the distribution of final seed ages.Dormancy in other organisms can also be studied using our model.One example is insect diapause [41], which is considered another example of bet-hedging.In many insect species, the larvae can enter diapause at a certain developmental stage to avoid unfavourable conditions, instead of proceeding with normal development to become adults.In a simple model of diapause [42], the larvae may undergo multiple years of diapause and have a fixed probability of (re)entering diapause each year (see fig. 1 of [42]), similar to Cohen's model of seed dormancy [11].This would correspond to our model with L = 1, such that the decision to enter diapause is memoryless.Another model assumes that the larvae can only undergo one period of diapause and must exit after that [43].This pattern is a special case of our model with L = 2, where the state s 0 would correspond to a new larva and s 1 to diapause.The larva can either develop to an adult with probability q 0 and produce offspring (arrow from s 0 back to itself ), or enter diapause with probability 1 − q 0 (arrow to s 1 ).However, once it undergoes diapause, it must exit and develop, so there is only one arrow leaving s 1 , which goes to s 0 with probability q 1 = 1.In this scenario, it was found that diapause is beneficial in varying environments that are temporally correlated [43], in agreement with our results.More generally, one may study situations where diapause can be repeated for a number of times, which would correspond to a diagram like figure 5a.Our results suggest that which form of diapause is evolutionarily favoured depends on the complexity of temporal structure in the environmental variation, which could be potentially tested in empirical studies.

Conclusion
We have shown that the internal states of organisms can serve as a memory to help the population adapt in varying environments.In order for this strategy to be useful, the environment must be temporally structured, and the internal states must become correlated with the environment.We have demonstrated that such correlation can arise from selection alone, without direct interaction with the environment.More generally, some internal states of organisms may be correlated with the environment as a result of phenotypic plasticity.For example, seeds produced in a good year may be bigger than those produced in a bad year, so seed size could provide a memory of the past environment.It is known that the seed size can affect germination probability [44], and it will be interesting to study if such dependence can benefit population growth in varying environments.
Organisms are complex systems with a lot of internal degrees of freedom, some of which might happen to become correlated with the environment through selection or plasticity.Even though these internal states might not have developed as sensors for environmental cues, they could be co-opted as information sources to guide the organism's behaviour.To test whether seed age could be co-opted to affect germination, one might compare accessions of annual plants in temporally structured environments and those in unpredictable environments.Our model predicts that the germination fraction would evolve to depend on the seed age in the former case, which can be tested in future experiments.Note that although we have focused on modelling the dependence of germination on seed age, in royalsocietypublishing.org/journal/rspb Proc.R. Soc.B 290: 20230096 reality, it can depend on both internal and external cues.Given any dependence on external cues, we expect that internal cues may provide additional information about the environment to help organisms adapt.The interaction between internal and external cues may be studied in more detail in future.
Dormancy has been proposed to cause a 'storage effect' that promotes species coexistence in varying environments [45].Our model of age-dependent dormancy may be studied in such community ecology context.If the presence of other species is viewed as part of the environment for the focal species, then internal states such as seed age could potentially provide a memory of past interaction with those other species.For example, reserve level of the predator may be an indicator of past encounters with prey [21].Historydependent ecological interactions have been experimentally indicated in microbial communities [46].It will be interesting to use our framework to study such ecological dynamics of organisms whose phenotypes depend on their memory.

Appendix A. Methods (a) Analytic derivation of Cohen's model
Consider a population of annual plant seeds, each of which can either germinate (ϕ = 1) or stay dormant (ϕ = 0) each year.The environment can be either good (1 ¼ 1) or bad (1 ¼ 0).If a seed germinates in a good year, it will reproduce and yield Y 1 number of seeds; but a seed germinating in a bad year will only yield Y 0 seeds, with Y 1 > Y 0 (in the main text we set Y 0 to 0 for simplicity).If a seed stays dormant, then the probability that it will remain viable is V.For Y 1 > V > Y 0 , it is favourable for a seed to germinate in a good year but stay dormant in a bad year.The number of seeds at year t is denoted by N t and obeys the equation: where 1 t is the environment in that year and q is the fraction of seeds that germinates.The number of seeds at year T can be calculated recursively as follows: where T 1 is the total number of years that the environment is 1.The long-term growth rate L is defined as the asymptotic rate of logarithmic increase: where P 1 ; lim T!1 ðT 1 =TÞ is the frequency of environment 1.The germination fraction q* that maximizes L is found by setting the derivative @L=@q to zero, which gives (assuming q* > 0): And the corresponding maximum growth rate L bet is: If the seeds have perfect information about the future environment, then they should all germinate in good years and stay dormant in bad years.This would result in a total population 1 instead of equation (A 2), which gives the maximum possible growth rate: The difference between L max and L bet is then given by: In the limit Y 0 → 0 and Y 1 ≫ V, it simplifies to: which is the entropy of the environment.
The aforementioned model can be generalized to include an external cue ξ that is correlated with the environment 1. Assume that, given ξ, the seeds will germinate with probability Pðf ¼ 1jjÞ ; q j .The total number of seeds then obeys the equation: where ξ t is the cue received in year t.Repeating the same procedure as earlier, one finds that the population after T years becomes: where T 1j is the number of years that the environment is 1, while the cue is ξ.The long-term growth rate is then given by: where P 1j ¼ lim T!1 ðT 1j =TÞ is the joint probability of the environment 1 and the cue ξ.The optimal germination fraction q Ã j that maximizes equation (A11) is given by (assuming q Ã j .0): which is the same as equation (A 4) except that P 1 is replaced by the conditional probability P 1jj ¼ P 1j =P j .The maximum growth rate achieved by using the external cue is then given by plugging equation (A 12) into equation (A 11), which gives: royalsocietypublishing.org/journal/rspb Proc.R. Soc.B 290: 20230096 The difference between L cue and L bet is then: which is precisely the mutual information between the environment 1 and the cue ξ.

(b) Numerical solution for age-dependent germination
In our model where the germination fraction depends on the seed age, neither the growth rate nor the optimal germination fraction has an analytic solution.Here, we describe how they are calculated numerically.Since the seeds are heterogeneous in age, the population is described by a vector N with components N a that represents the number of seeds of age α.As described in the main text, the vector N t at year t obeys the equation: where the matrix M depends on the current environment 1 t and the germination fractions q a ; Pðf ¼ 1jaÞ, as given in equation (3.1).Thus, the population vector after a long time T is expressed as follows: and the long-term growth rate is formally given by the largest Lyapunov exponent of the product of matrices: where | • | is the matrix norm, which we choose to define as the largest eigenvalue for non-negative matrices.Compared to Cohen's model, here L cannot be calculated analytically because the matrix multiplications are non-commutative.To numerically calculate L, we simply use equation (A17) with a very large T, as the limit is expected to converge [35].
We first draw a sequence of T random environments as follows.Define an epoch of time t 1 as the number of consecutive years that the environment remains to be 1 until it switches.The good and bad epochs are drawn from the distributions: and Here, m 1 is the mean duration for the epochs, σ characterizes the variability of the bad epochs and Z is a normalization constant.
For the example used in the main text (figure 1b), μ 1 = μ 0 = 5 and σ = 2.A total of 50 000 epochs are drawn for each environment, with a total length T ≈ 500 000.
We normalize M (t) at every time step by the value of its largest entry, and this normalization factor n t is stored.The Lyapunov exponent is then given by L ¼ ð1=TÞ ð where w is the largest eigenvalue of the normalized M (T ) (which does not matter for L when T is large, but matters for its derivative that we calculate below).To find the germination fractions q Ã a that maximizes L, we use the optimization routine L-BFGS-B, which allows us to impose the constraint 0 q Ã a 1.Besides the numerical function that calculates L as described earlier, we also supply the Jacobian of the function, i.e. the derivative @L=@q a .This requires calculating the derivative of M (T ) with respect to q a , which can be done using the recursive relation: together with that for M (t) in equation (A 20), from t = 1 all the way to T. We normalize @M ðtÞ =@q a by the same factor n t as for M (t) at every time step.The derivative of L is then given by @L @q a ¼ 1 T 1 jM ðTÞ j @jM ðTÞ j @q a ¼ 1 T where u and v are the left and right eigenvectors of M (T ) corresponding to its largest eigenvalue w, respectively.This derivative is then supplied as the Jacobian to an optimization routine to find the optimal q* that maximizes L. For numerical optimization, we use the SciPy function optimize.minimize()with the L-BFGS-B method (limited-memory Broyden-Fletcher-Goldfarb-Shanno algorithm with bounds) [48].

(c) Simulating a lineage
Simulation of a continuous lineage of seeds is used to estimate the joint probability Pð1 t , a tÀ1 Þ of the environment 1 t and the seed age α t−1 in §2(b), which is then used to calculate their mutual information Ið1 t ; a tÀ1 Þ in §4(b).For a given set of germination fractions q a , the simulation is done as follows.We start from a fresh seed of age 0. The sequence of environments, f1 1 , . . ., 1 T g, is drawn beforehand as described in §A(b).
In each year, we decide whether the seed germinates using the germination probability that corresponds to its age.To account for selection bias, we weight the probabilities by the fitness values in the current environment.That is, in year t, the seed along the lineage has probability to germinate and reset the age to 0, and otherwise stays dormant with its age increased from α t−1 to α t = α t−1 + 1.We repeat this procedure from t = 1 to T, recording the sequence of α t .Afterwards, the number of times that the pair ð1 t , a tÀ1 Þ takes a particular combination of values is counted, which is then normalized to be the joint probability distribution Pð1 t , a tÀ1 Þ, from which the mutual information Ið1 t ; a tÀ1 Þ is calculated.Lineage simulation is also used to calculate the distribution of dormancy duration in §4(a), i.e. the distribution of how many consecutive years a seed stays dormant in the absence of environmental variation.To calculate this distribution, we once again start with a fresh seed of age 0 and use the probability q 0 to decide if the seed germinates.This time the probability is not weighted by the fitness because we are calculating the dormancy durations in the absence of selection.The aforementioned procedure is repeated for a long period of time T, and the sequence of phenotypes at each time step is recorded as ϕ t .The duration of germination or dormancy is calculated by parsing the sequence of phenotypes {ϕ t } into royalsocietypublishing.org/journal/rspbProc.R. Soc.B 290: 20230096 consecutive epochs of germination or dormancy.The distribution of their durations is then calculated by normalizing the histograms of these epochs.Note that these distributions can also be calculated using equation (A 31) in appendix §A(e).

(d) Exhaustive search of state diagrams
To verify that the age diagram in figure 5a is the optimal topology, we test all possible state diagrams for up to six internal states.For a diagram with L states, we label the states as s 0 , s 1 , …, s L−1 .Each state has two outgoing arrows, corresponding to either dormancy or germination.Each arrow can go to any other state or loop back.Therefore, naively, there can be L 2L possible diagrams.However, many of these diagrams are equivalent in the sense that they are simply permutations of the states.To remove the redundant diagrams, we use a 'sieve' method as follows.We first represent a diagram by a (L × 2) integer matrix, whose entry of the αth row and ϕth column represents which state the system will transition to if it is at age α and expresses phenotype ϕ.The diagrams are then indexed by a number that results from flattening the matrix and treating it as a base-L number.Then, we enumerate all L 2L diagrams starting from the index 0.For each diagram, we find all its permutations and remove their indices from the list.Furthermore, we exclude diagrams that have two or more disjoint parts to keep only connected diagrams.We go over the list of diagrams, skipping the indices that have been removed.In the end, the total number of nondegenerate diagrams for L = 1, 2, … is nðLÞ ¼ 1, 6, 52, 892, 21 291, 658 885, . . ., which is the number of unlabeled, strongly connected, L-state, two-input automata (Sequence A027835 in The On-Line Encyclopedia of Integer Sequences (OEIS) [49]).This number grows quickly, and we are only able to study diagrams for up to L = 6.
For each of the diagrams with L states, we numerically find the optimal q a and the maximum L as in §A(b).This is computationally intensive and is done on a computer cluster.Then, among all diagrams of L states, we find the optimal diagram with the largest L. For up to L = 6, it turns out that the age diagram is the optimal diagram for our model.

(e) Analytical results for extreme selection
In the limit of extreme selection, the fitness matrix is diagonal, i.e. f 1f ¼ V 0 0 Y .This means, hypothetically, that a seed can survive only if it germinates in a good year or stays dormant in a bad year.In this case, the long-term growth rate L and the optimal germination fractions q Ã a have analytical solutions.Indeed, the population becomes homogeneous because, once it encounters a good year, only the seeds that germinate will survive, and subsequently the population will consist of only fresh seeds.From then on, the seed age will be synchronized with the number of consecutive bad years and will be reset to 0 whenever there is a good year.Let β t−1 be the number of consecutive bad years right before year t (which is 0 if the previous year is good).It will be equal to the seed age α t−1 of the population at the beginning of year t.Therefore, the seed population changes over time according to: depending on whether the environment 1 t ¼ 0 or 1.Over a period of time T, the number of seeds will be: where T 1b is the number of years that the environment is 1, while the previous number of consecutive bad years is β.This equation has the same form as equation (A 10), with the external cue ξ replaced by β.The long-term growth rate has the expression where P 1b ¼ lim T!1 ðT 1b =TÞ is the joint probability of the environment 1 t and the number of bad years β t−1 .Setting the derivative @L=@q a ¼ 0, the optimal germination fractions q Ã a are found to be q Ã a ¼ P 1a P 0a þ P 1a ; P 1ja ; Pð1 t ¼ 1jb tÀ1 ¼ aÞ: ðA 26Þ Here, Pð1 t ¼ 1jb tÀ1 ¼ aÞ represents the conditional probability that the coming year is good, given that there has been α consecutive bad years.It is related to the duration distribution of bad years, P(τ 0 ) from equation (A19), through Substituting equations (A 26) into (A 25) gives the maximum growth rate using internal memory: where L bet ¼ P 0 log P 0 V þ P 1 log P 1 Y in this extreme selection limit, and P 1 is the frequency of the environment.Note that β t−1 in this case is the sufficient statistic for the past environmental sequence.Therefore, the benefit of internal memory can be written as L mem À L bet ¼ Ið1 t ; f1 s g s,t Þ.This expression is true for any type of environmental process in the extreme selection limit; similarly, for any type of state diagrams with internal states α t , the attainable long-term growth rate will be given by L int ¼ L bet þ Ið1 t ; a tÀ1 Þ [32].An important consequence of equation (A 26) is that, for the germination fractions q Ã a , the dormancy duration of the seeds (as shown in figure 7b) will have the same distribution as the duration of bad years (figure 1b).This is because, by definition, q a ; Pðf t ¼ 1ja tÀ1 ¼ aÞ.Let δ 0 denote the duration of dormancy, then similar to equation (A 27), we have

Figure 1 .
Figure 1.(a) Schematic illustration of Cohen's model of seed dormancy in annual plants.Each year may be good or bad for plant growth.A seed can either germinate to produce a yield Y 1 that depends on the environmental condition 1, or stay dormant with a probability V of still being viable next year.The number of seeds at the end of year t is N t .The parameter values used in our calculations are Y 0 = 0, Y 1 = 4 and V = 0.9.(b) The distribution of duration of consecutive good years and bad years.We choose the duration of good years to follow a geometric distribution with a mean of 5, and the duration of bad years to have a Gaussian distribution with a mean and standard deviation of 5 ± 2 cut off at 0 and 10.(c) A state diagram that represents the seed age.Each state s a represents a seed of age α.Blue arrows represent dormancy that increases the age by 1; orange arrows represent germination that may produce new seeds of age 0. Weights on the arrows represent the probability of germination or dormancy.

Figure 5 .Figure 6 .
Figure 5. State diagrams for age-dependent germination.(a) The germination fraction q depends on the seed age α up to α = L − 1, beyond which it remains the same.Varying the length L effectively varies the memory capacity of the organisms.(b) With only one state (L = 1), the organism effectively has no memory, and the germination fraction is a constant, corresponding to simple bet-hedging.(c) The two-state case corresponds to a Markov process where the organisms switch back and forth between two phenotypes, with transition probabilities P(ϕ 1 |ϕ 0 ) = q 1 and P(ϕ 0 |ϕ 1 ) = 1 − q 0 .

Figure 7
Figure 7.The distribution of the duration of consecutive germinations or dormant years along a lineage of seeds.Different colours correspond to age-dependent germination fractions q a for different memory capacities L. (a) For each L, the duration of germinations matches a geometric distribution with a mean of 1/q 0 (dashed line for L = 2 and solid line for L = 10), meaning that there is no memory of previous germinations.(b) The duration of dormancy has a distribution that changes shape depending on the memory capacity L. L = 2 ( phenotypic switching) results in a geometric distribution with a mean of 1/(1 − q 1 ) (dashed line).Larger L results in deviation from a geometric distribution, which is indicative of having internal memory.
Figure3.Probability of the coming environment 1 t conditioned on the seed age α t−1 at the beginning of year t, as calculated by simulating a lineage of seeds.Dashed line is the marginal probability of the environment, which would indicate that the seed age is uncorrelated with the environment.Blue bars are when the population uses a bet-hedging strategy with a constant germination fraction.Orange bars are when the germination fraction depends on the seed age to maximize population growth rate.In both cases, the seed age is correlated with the environment and thus useful as an internal cue.Dependence of the germination fraction q on the seed age α that maximizes the population growth rate.Blue bars are when the environment is temporally structured, as described by the duration of good and bad years in figure1b.Orange bars are when the environment is drawn independently each year, for which the germination fraction need not depend on seed age and is equal to the bet-hedging solution in equation (2.2) (dashed).
royalsocietypublishing.org/journal/rspb Proc.R. Soc.B 290: 20230096 can remember, the better they may predict and adapt to the environment (to the extent that the environment is predictable).To test this in our model, we can vary the memory size by changing the number of possible internal states.The state diagram in figure To calculate L, we need to calculate the product Q T t¼1 Mð1 t ; qÞ.For convenience, we define M ðsÞ ; t ; qÞ.Then M (T ) can be calculated recursively by M ðtÞ ¼ Mð1 t ; qÞ Á M ðtÀ1Þ : ðA Pðf t ¼ 1ja tÀ1 ¼ aÞ ¼ .org/journal/rspbProc.R. Soc.B 290: 20230096 0 ¼ aÞ ¼ Pðt 0 ¼ aÞ: ðA 30Þ Incidentally, for a general q a , it can be shown thatPðd 0 ¼ aÞ ¼ q a Y aÀ1 k¼1 ð1 À q k Þ: ðA31Þroyalsocietypublishing